ChatGPT vs state-of-the-art models: a benchmarking study in keyphrase generation task

R. Marti­nez-Cruz, A.J. López López, J. Portela

Applied Intelligence Vol. 55, pp. 50-1 - 50-25

Resumen:

Transformer-based language models, including ChatGPT, have demonstrated exceptional performance in various natural language generation tasks. However, there has been limited research evaluating ChatGPT’s keyphrase generation ability, which involves identifying informative phrases that accurately reflect a document’s content. This study seeks to address this gap by comparing ChatGPT’s keyphrase generation performance with state-of-the-art models, while also testing its potential as a solution for two significant challenges in the field: domain adaptation and keyphrase generation from long documents. We conducted experiments on eight publicly available datasets spanning scientific, news, and biomedical domains, analyzing performance across both short and long documents. Our results show that ChatGPT outperforms current state-of-the-art models in all tested datasets and environments, generating high-quality keyphrases that adapt well to diverse domains and document lengths.

Palabras Clave: ChatGPT · Text generation · Keyphrase generation · Natural language processing · Deep learning · Domain adaptation · Long documents · Large language models

Índice de impacto JCR y cuartil WoS: 3,400 - Q2 (2023)

Referencia DOI: https://doi.org/10.1007/s10489-024-05901-4

Publicado on-line: Noviembre 2024.

Cita:
R. Marti­nez-Cruz, A.J. López López, J. Portela, ChatGPT vs state-of-the-art models: a benchmarking study in keyphrase generation task. Applied Intelligence. Vol. 55, pp. 50-1 - 50-25 [Online: Noviembre 2024]